This invention relates generally to a multi-purpose portable imaging device, and more particularly to a device for displaying images from sensors embedded in a hand-held device, which performs various specialized functions which may be determined by hardware and software components.
Algorithms for processing digital images are well known and are presented in such literature as Digital Image Processing, by Rafael G. Gonzalez and Richard Woods, Addison-Wesley Publishing Company, Inc., 1992, herein incorporated by reference. The book gives examples for image acquisition, storage, processing, communication, and display.
Specialized devices for collecting, processing, and displaying data have been developed for various applications. Examples of electromagnetic spectrum devices which are well known in the art are radar systems, x-ray systems, magnetic resonance imaging (MRI) systems, and infrared (IR) systems. An example of a well-known device using sound waves is an ultrasound system. An example of a well-known device for use in water is sonar. Another example is an airport security baggage x-ray device. Most of these devices have the disadvantages of being costly, occupying a large physical space, consuming large amounts of power, and being limited to performing a single, dedicated task.
An example of a X-ray device for imaging is described in U.S. Pat. No. 5,181,234 entitled, xe2x80x9cX-Ray Backscatter Detection Systemxe2x80x9d, issued to Smith on Jan. 19, 1993, and incorporated herein by reference. It discloses an x-ray scanner for the human body to detect hidden weapons. The commercial implementation of the invention is the SECURE 1000 from Rapiscan Security Products, Inc. with a list price of $110,000.
Because of advances in miniaturization and cost-reduction in the semiconductor art, certain devices for collecting, processing, and displaying data have become small enough to be portable or wearable. These compact devices typically comprise sensors for collecting data, a processor for manipulating data, and a graphical display for showing real-time information.
For example, in the radar art U.S. Pat. No. 4,641,317 entitled xe2x80x9cSpread Spectrum Radio Transmission System,xe2x80x9d issued to Larry Fullerton on Feb. 3, 1987, and incorporated herein by reference, discloses a communication system which uses an intelligence signal to modulate the spacing of narrow pulses of radio transmission which is essentially non-interfering. Additionally, U.S. Pat. No. 5,668,555 entitled, xe2x80x9cImaging System and Apparatus,xe2x80x9d issued to Starr on Sep. 16, 1997, and incorporated herein by reference, discloses a low-cost, portable radar system. The ""555 patent is based upon the advance in the field of miniaturization of radar circuits as disclosed in U.S. Pat. No. 5,361,070 entitled, xe2x80x9cUltra-Short Pulse Generator,xe2x80x9d issued to McEwan on Dec. 28, 1993, incorporated herein by reference, which discloses a radar on a chip. The ""555 patent receives radar data which is in turn loaded into a xe2x80x9cCADxe2x80x9d (Computer Aided Design) program, which in turn generates a computer image from the radar data. However, such a system, using CAD technology, would not generate images in real time in a portable device, as the amount of processing power required to render images in CAD format is considerable.
An example is the visible art is U.S. Pat. No. 5,712,682. incorporated herein by reference entitled, xe2x80x9cCamera having an Adaptive Gain Control,xe2x80x9d issued to Hannah on Jan. 27, 1998, which discloses an imager with gain control signal for adjusting the level of gain applied by an amplifier to a digital output signal.
Another example in the infrared art is U.S. Pat. No. 5,675,149 entitled, xe2x80x9cCompact Thermal Cameraxe2x80x9d, issued to Wood et al. on Oct. 7, 1997, which discloses a low-cost, hand-held infrared camera.
An example of radiation detection is provided by U.S. Pat. No. 5,707,879 entitled, xe2x80x9cNeutron Detector Based on Semiconductor Materials,xe2x80x9d issued to Karl Reinitz on Jan. 13, 1998, which discloses a radiation detector on a semiconductor chip.
The current generation of imaging devices has three major disadvantages. First, the current generation of imaging devices has the disadvantage of being large and costly requiring external power to operate. Second, the current generation of imaging devices has the disadvantage of being limited to only one kind of sensor, such as radar, CCD, Infrared and the like. This limitation is determined by the lack of adequate processing power of the computer technology that is available, which makes it impractical in a portable device to process data from more than one kind of sensor. For example, using current technology is not feasible to combine an IR sensor with a radar sensor into one compact device. Combining data from multiple sensors is known in the art as sensor fusion. Third, the current generation of imaging devices has the disadvantage limited to performing only one task, such as radar imaging for blood flow visualization. Using the previous example, if instead of blood flow visualization, radar imaging for tissue analysis is desired, another device must be constructed.
Although the prior art teaches about ways to accomplish sensor fusion, the improvements are limited to large devices. Information on sensor fusion can be found in literature such as Multi-Sensor Fusion, by Richard R. Brooks and S. S. Iyengar, Prentice Hall, 1998 at www.phptr.com incorporated herein by reference. An example of sensor fusion is provided by U.S. Pat. No. 5,274,236 entitled, xe2x80x9cMethod and Apparatus for Registering Two Images from Different Sensors,xe2x80x9d issued to Pascale et al. on Dec. 28, 1993, and incorporated herein by reference. The ""236 patent discloses an invention that improves delivery of a missile to a target by fusing data from an infrared sensor on an aircraft with data from a forward-looking infrared sensor on a guided missile. U.S. Pat. No. 5,531,227 entitled, xe2x80x9cImaging Device and Method,xe2x80x9d issued the Schneider on July 2, 1996, shows the use of different mechanisms to obtain an image by using image libraries and is incorporated herein by reference.
Addressing the disadvantage of being limited to only one task, an object of one embodiment of the present invention is to provide a means for easily changing the software of the invention so that the same device may be used for a different application. To this end, the invention has a means for receiving a cartridge with memory modules, or other storage media containing software. For example, one cartridge may contain software for use in medical imaging while another cartridge may contain software for use in traffic accident investigations. These cartridges process the data from some or all of the sensors on the device, but manipulate the data for a specialized result such as customized display with highlights. The present invention maximizes value by leaving the hardware of the compact imaging device essentially unchanged, while simultaneously allowing the function of the invention to change by replacing a software cartridge.
Addressing the disadvantage of being limited to only one kind of sensor, it is an object of an embodiment of the present invention to provide information from a variety of sensors to a graphical display on a portable device. These sensors comprise, but are not limited to radar transmitters and receivers, lasers, receivers of various electromagnetic spectrum such as Infrared or ultra-violet, CCD cameras, and navigational/position technologies such as Global Positioning System (GPS).
An example of 3D detection of an object within a static image using CCD cameras is provided by U.S. Pat. No. 5,877,803 entitled, xe2x80x9c3-D Image Detector,xe2x80x9d issued to Wee et al on Mar. 2, 1999, and incorporated herein by reference. Wee uses multiple CCD cameras to obtain data representing the magnitude of light impinging upon objects in a field of view. Triangulation algorithms commonly known in the mathematical art are then applied to the data to derive surface depth and contour information.
The present invention may be implemented utilizing a low power broadband radar such as the Micro-power Impulse Radar (MIR) technology developed by Lawrence Livermore National Laboratories. Examples of such MIR applications may be found in U.S. Pat. Nos. 5,457,394, 5,465,094, 5,479,120, 5,510,800, 5,512,834, 5,519,400, 5,521,600, 5,581,256, 5,589,838, 5,609,059, and 5,610,611, incorporated herein by reference. The MIR devices comprises small, low power, broadband radar devices which are being developed for a wide range of applications. These radar devices are coupled to antenna arrays and a processor to form a complete radar imaging system. Hardware and software is used to reconstruct 2D and 3D views of the scene.
Due to their low cost and size, numerous MIR sensors may be assembled into arrays for synthetic and real aperture image formation in 2-D and 3-D. Radar return signals are digitized and stored in a lap-top computer. Reconstruction of cross-sectional images from B-scan or waterfall type data is performed by diffraction topography software on the lap-top. Images of the scene are displayed directly on a screen within ten seconds (in 2-D). However, such slow imaging response times may be unacceptable for many applications, and moreover limit the overall usefulness of MIR technology.
For example, if such images could be processed in real-time, an animated image may be produced. Such animated images may have many applications. For example, an animated image (real-time image) may allow a doctor to view blood flow through a patient or other internal workings, rather than a static image.
For portable applications, an imager may be xe2x80x9csweptxe2x80x9d through an area and the user may viewxe2x80x94in real timexe2x80x94the corresponding image. Thus, for example, such an imager may be used to find underground objects (e.g., pipes, ducts, wiring, and the like) by sweeping such a device over a particular area and viewing the resultant image. However, such applications require near real-time processing of sensor data.
Such improvements in the art may be made possible by the exceptional processing power inherent in using an integrated processor array. An integrated processor array is an innovation in computer technology which provides fast and inexpensive computer power in a compact space and is further described below.
The present invention is a multi-purpose portable imaging device. The device is small enough to be hand-held or wearable and has embedded on its surface at least one sensor. These sensors may be active or passive. Analog energy received from the sensors is converted into a digital format and sent to an advanced computer.
The computer is constructed on a parallel array platform such as shown in U.S. Pat. No. 5,625,836 Barker et al., incorporated herein by reference. The computer has the capability of taking data from multiple sensors and providing sensor fusion features. The data is processed and displayed in a graphical format in real time which is viewed on the imaging device. A keypad, or touch screen, or other entry device for entering data and commands may be available on the device. The device has the capability of using a removable cartridge embedded with memory modules containing application software for manipulating data from the sensors and RAM or peripherals such as GPS units. The data may also be uploaded to other computers.